Systems and methods for digital entertainment

ABSTRACT

Entertainment-based methods and apparatus involving a video display facility and a lighting facility. In one example, lighting effects generated by the lighting facility are automatically coordinated with the video display facility. In another example, the lighting effects generated by the lighting facility are automatically coordinated with a video signal provided to the video display facility.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. §120 as acontinuation (CON) of U.S. Non-provisional application Ser. No.10/045,604, filed Oct. 23, 2001, entitled “Systems and Methods forDigital Entertainment.”

Ser. No. 10/045,604 in turn claims the benefit under 35 U.S.C. §119(e)of the following United States provisional patent applications:

-   -   Ser. No. 60/243,250, filed Oct. 25, 2000, entitled “Illumination        of Liquids;”    -   Ser. No. 60/262,153, filed Jan. 17, 2001, entitled “Information        Systems;”    -   Ser. No. 60/242,484, filed Oct. 23, 2000, entitled “Systems and        Methods for Digital Entertainment;”    -   Ser. No. 60/262,022, filed Jan. 16, 2001, entitled “Color        Changing LCD Screens;”    -   Ser. No. 60/268,259, filed Feb. 13, 2001, entitled “LED-Based        Lighting Systems and Methods for Vehicles;” and    -   Ser. No. 60/277,911, filed Mar. 22, 2001, entitled “Systems and        Methods for Digital Entertainment.”

Ser. No. 10/045,604 also claims the benefit under 35 U.S.C. §120 as acontinuation-in-part (CIP) of the following United States PatentApplications:

-   -   Ser. No. 09/213,607, filed Dec. 17, 1998, entitled “Systems and        Methods for Sensor-Responsive Illumination;”    -   Ser. No. 09/213,189, filed Dec. 17, 1998, entitled “Precision        Illumination Methods and Systems,” now U.S. Pat. No. 6,459,919;    -   Ser. No. 09/213,581, filed Dec. 17, 1998, entitled “Kinetic        Illumination Methods and Systems;”    -   Ser. No. 09/213,540, filed Dec. 17, 1998, entitled “Data        Delivery Track,” now U.S. Pat. No. 6,720,745    -   Ser. No. 09/333,739, filed Jun. 15, 1999, entitled “Diffuse        Illumination Methods and Systems;”    -   Ser. No. 09/626,905, filed Jun. 27, 2000, entitled “Illumination        Components,” now U.S. Pat. No. 6,340,868, issued Jan. 22, 2002;    -   Ser. No. 09/742,017, filed Dec. 20, 2000, entitled “Lighting        Entertainment System,” which is a continuation of U.S. Ser. No.        09/213,548, filed Dec. 17, 1998, now U.S. Pat. No. 6,166,496,        issued Dec. 26, 2000;    -   Ser. No. 09/815,418, filed Mar. 22, 2001, entitled “Lighting        Entertainment System,” now U.S. Pat. No. 6,577,080, which also        is a continuation of U.S. Ser. No. 09/213,548, filed Dec. 17,        1998, now Patent No. 6,166,496, issued Dec. 26, 2000;    -   Ser. No. 09/616,214, filed Jul. 14, 2000, entitled “Systems and        Methods for Authoring Lighting Sequences;”    -   Ser. No. 09/805,368, filed Mar. 13, 2001, entitled “Light        Emitting Diode Based Products;”    -   Ser. No. 09/805,590, filed Mar. 13, 2001, entitled “Light        Emitting Diode Based Products;”    -   Ser. No. 09/917,246, entitled “Systems and Methods for Color        Changing Device and Enclosure,” filed Jul. 27, 2001;    -   Ser. No. 09/923,223, entitled “Ultraviolet Light Emitting Diode        Systems and Methods,” filed Aug. 6, 2001;    -   Ser. No. 09/886,958, entitled “Method and Apparatus for        Controlling a Lighting System in Response to an Audio Input,”        filed Jun. 21, 2001; and    -   Ser. No. 09/215,624, filed Dec. 17, 1998, entitled “Smart Light        Bulb,” now U.S. Pat. No. 6,528,954, which in turn claims the        benefit under 35 U.S.C. § 119(e) of the following U.S.        Provisional applications:        -   Ser. No. 60/071,281, filed Dec. 17, 1997, entitled            “Digitally Controlled Light Emitting Diodes Systems and            Methods;”        -   Ser. No. 60/068,792, filed Dec. 24, 1997, entitled            “Multi-Color Intelligent Lighting;”        -   Ser. No. 60/078,861, filed Mar. 20, 1998, entitled “Digital            Lighting Systems;”        -   Ser. No. 60/079,285, filed Mar. 25, 1998, entitled “System            and Method for Controlled Illumination;” and        -   Ser. No. 60/090,920, filed Jun. 26, 1998, entitled “Methods            for Software Driven Generation of Multiple Simultaneous High            Speed Pulse Width Modulated Signals.”

Each of the foregoing applications is hereby incorporated herein byreference.

BACKGROUND

Computer games are well known, wherein one or more users interact with acomputer to play a game, typically involving use of a control device,such as a mouse, joystick or keypad, to move objects that appear on adisplay screen to accomplish objectives of the game. There are manytypes of games, including first-person games shooting, strategy games,war games, fighting games, puzzles, and many others. Computer games areplayed on or use many devices, including televisions, consoles, PDAs,handheld game devices, personal and laptop computers, and others. Somegames are run on standalone computers, while others employ networks,such as the Internet and World Wide Web.

The marketplace for computer games is very competitive, and improvementsto games are actively sought. Past improvements include enhancements tothe graphical or sound quality of games, improvements to systems thatrun games, such as processing speeds, and introduction of innovativegame types. A need exists for further improvements of the computer gameexperience, including enhancements that take advantage ofcharacteristics of the environment of the game user.

SUMMARY

The present disclosure sets forth improvements to computer games andother computer applications through the coordinated control of lightingsystems that illuminate the environment of the user in coordination withthe play of a game or the use of an application. In embodiments, thelighting systems disclosed herein change the illumination of the user'sreal world environment in coordination with events, attributes andobjects of a computer game.

In an embodiment there is disclosed a system for imparting informationusing a lighting system. The system includes a computing deviceincluding a video display; a lighting system in communication with thecomputing device for producing illumination; and a software applicationfor dynamically controlling the illumination in response to or incoordination with information presented on the video display so as toimpart information to a user. The software application can include agame and/or objects and may allow for relation of a portion of thelighting system to an object. The information provided by theillumination system can relate to an attribute of an object. Thesoftware application can be controlled by a second software applicationwhich can also include a game.

In an embodiment, the lighting system can include an LED and/or caninclude a screen upon which the illumination is projected. This screencan be a cabana. The computing system may also include a video gameconsole.

In an embodiment, the information provided by the lighting system can bedifferent from the information provided on the video display, either bybeing additive to the information provided on the video display, orduplicative of the information provided on the video display but in adifferent form. In an embodiment, the video display could provide noinformation.

In an embodiment there is disclosed a system for controlling a lightingsystem comprising: lighting system including a lighting fixture, thelighting system in communication with a computing device; and a softwareapplication including a software object operating in conjunction withthe computing device; wherein, the software application attaches thecontrol of the lighting fixture to the software object. The softwareapplication may include a game and the software object may comprise anobject in the game. The system could also include a library of effectsfor use with the lighting system.

In an embodiment there is disclosed a screen for use with a lightingsystem comprising; a frame designed to be placed in proximity to theuser of a computing system and; a material mounted on the frame; whereinthe material is arranged in a manner so as to be able to reflectillumination produced by a lighting system to the user of the computingsystem. This screen may be shaped to form a cabana and/or a portion of asphere and/or may be formed so as to be repeatedly assembled anddisassembled. Further, the computing system may include a video displayand the frame may be designed to be placed to at least partially enclosethe video display, be placed behind the video display relative to theuser, and/or at least partially encloses the computing system and theuser.

In an embodiment the screen could further include a mounting bar for theattachment of lighting fixtures to the screen. The mounting bar may bearranged so that the lighting fixtures have a fixed point of attachmentto the mounting bar. The mounting bar may alternatively or additively bearranged so that fixtures have a fixed point of projection onto thescreen when attached to the mounting bar. Those lighting fixtures maycomprise at least a portion of the lighting system.

In an embodiment there is disclosed a software application for use on acomputing device comprising: computer code for generating a computergame on a computing device; and computer code for controlling a lightingsystem in communication with the computing device.

In another embodiment there is disclosed a method for visualizing therelative location of virtual objects within a virtual environmentcomprising: having a computing device; generating a virtual environmenton the computing device, the virtual environment containing a pluralityof virtual objects; associating with a virtual object, the illuminationfrom a lighting fixture; and visualizing the relative location of thevirtual object by the positioning of the illumination. The visualizingmay include the position of the illumination corresponding to theposition of the lighting fixture or the position of the illuminationcorresponding to the position on a surface which is illuminated by theillumination. The position on the surface may perform at least one ofthe following: reflection of the illumination, refraction of theillumination, absorption and reemission of the illumination.

In another embodiment there is disclosed a method for enhancing the playof a computer game comprising: providing to a user a lighting system;providing to the user software for controlling the lighting system, thesoftware being capable of interfacing with a computer game; and allowingthe user to use the software to control the lighting system in a mannerthat enhances the play of the computer game.

In another embodiment there is disclosed a lighting system for use witha software application comprising: at least one lighting fixture; and acomputer application compatible with the software application, thecomputer application allowing for the software application to provideinformation to a user through illumination generated by the lightingfixture. The computer application may include computer software and/orcomputer hardware. The lighting fixture may include an LED and/or may beone of a plurality of networked lighting fixtures. The lighting systemmay further include a mounting apparatus for holding the lightingfixture and/or a surface for the reflection of illumination.

In a still further embodiment there is disclosed a method for allowing asoftware developer to include control of a lighting system within asoftware application comprising: providing a lighting systemsubstantially similar to one provided to a user to a software developer;and providing an interface for allowing the lighting system tocommunicate with a computing device to a software developer; wherein thesoftware developer can use the interface to include control for thelighting system within a software application. There can also beprovided a library of prebuilt effects that can be used to generate aparticular lighting effect on the lighting system to the softwaredeveloper.

BRIEF DESCRIPTION OF DRAWINGS

The foregoing and other objects and advantages of the invention will beappreciated more fully from the following further description thereof,with reference to the accompanying drawings, wherein:

FIG. 1 shows a computer user with an embodiment of a lighting system.

FIG. 2 shows an embodiment of a cabana for use with a lighting system.

FIG. 3 shows an embodiment of a video display with a lighting systembuilt in for displaying illumination indirectly to a user.

FIG. 4 shows an embodiment of speakers with a lighting system built in.

FIG. 5 shows an embodiment of a lighting system which could be usedherein.

FIG. 6 shows an embodiment of a video display with a lighting systembuilt in for displaying illumination directly to a user.

FIG. 7 shows a flow diagram for a method of coordinating illumination ofan environment with execution of content of a computer application.

FIG. 8 shows a flow diagram for a method of adding lighting as aninstance of an object-oriented programming technique used to code acomputer game.

FIG. 9 shows a flow diagram for a method of mapping real world lightsand virtual lights to facilitate the coordinated control of lightingwith the execution of a computer application.

FIG. 10 shows the schematic of a system for storing files for the codingof an application for control of lighting in coordination with executionof a computer game.

FIG. 1I shows a file structure for a configuration file for a methoddepicted in connection with FIG. 9.

FIG. 12 shows a flow diagram for creation and use of a configurationfile for coordinated illumination.

FIG. 13 shows an embodiment of coordinated illumination wherein aportion of a real world environment is lit by an array of light emittingdiodes.

FIG. 14 shows a flow diagram for mapping of house lights to lights in avirtual lighting environment.

FIG. 15 shows a flow diagram for coordinated control of real worldlights that are mapped to lights in a virtual lighting environment.

FIG. 16 shows a system for coordinating lighting with a computerapplication for simulating an architectural project.

DETAILED DESCRIPTION

To provide an overall understanding of the invention, certainillustrative embodiments will now be described, including variousapplications for programmable LED's and LED lighting systems. However,it will be understood by those of ordinary skill in the art that themethods and systems described herein may be suitably adapted to otherenvironments where programmable lighting may be desired, and embodimentsdescribed herein may be suitable to non-LED based lighting. Thedescriptions below focus primarily on using LED lighting systems forenhancement of computer games as that term would be understood by one ofskill in the art. In particular, the below embodiments focus primarilyon a “first person” type of interactive game involving space battleswhere the software controlling the game is present on a user's computer,for example installed on the user's hard drive or on a CD ROM or otherstorage media controlled by the user. One example of such a computergame is produced by Mad Dog software under the title “Star Trek:Armada.” This type of computer game represents only one type of computergame with which the below described systems and methods can be used. Oneof skill in the art would readily see how to apply the below-describedembodiments to other types of computer games. Further these games neednot be present on the user's computer but could be run off of a networksuch as, but not limited to, the World Wide Web, the Internet, or anyextranet or intranet network, or could be console-type orvideo-parlor-type computer or video games. In addition, one of skill inthe art would understand that the embodiments described below could beused in conjunction with any type of computer software that need not bea game, but of any type of computer application. Further, the user neednot be operating a computer, but could be operating any type ofcomputing device, capable of running a software application that isproviding that user with information.

In computer games, there is typically a display screen (which could be apersonal computer screen, television screen, laptop screen, handheld,gameboy screen, computer monitor, flat screen display, LCD display, PDAscreen, or other display) that represents a virtual world of some type.There is also typically a user in a real world environment thatsurrounds the display screen. The present invention relates to computergames and their surrounding environment.

In an embodiment of the invention described herein, the environment of auser of a computer game includes one or more light systems. As usedherein “light systems” should be understood where context is appropriateto comprise all light systems, including LED systems, as well asincandescent sources, including filament lamps, pyro-luminescentsources, such as flames, candle-luminescent sources, such as gas mantlesand carbon arc radiation sources, as well as photo-luminescent sources,including gaseous discharges, fluorescent sources, phosphorescencesources, lasers, electro-luminescent sources, such aselectro-luminescent lamps, light emitting diodes, and cathodeluminescent sources using electronic satiation, as well as miscellaneousluminescent sources including galvano-luminescent sources,crystallo-luminescent sources, kine-luminescent sources,thermo-luminescent sources, triboluminescent sources, sonoluminescentsources, and radioluminescent sources. Light systems may also includeluminescent polymers capable of producing primary colors.

As used herein, the term “LED” means any system that is capable ofreceiving an electrical signal and producing a color of light inresponse to the signal. Thus, the term “LED” should be understood toinclude light emitting diodes of all types, light emitting polymers,semiconductor dies that produce light in response to current, organicLEDs, electro-luminescent strips, and other such systems. In anembodiment, an “LED” may refer to a single light emitting diode havingmultiple semiconductor dies that are individually controlled. It shouldalso be understood that the term “LED” does not restrict the packagetype of the LED. The term “LED” includes packaged LEDs, non-packagedLEDs, surface mount LEDs, chip on board LEDs and LEDs of all otherconfigurations. The term “LED” also includes LEDs packaged or associatedwith phosphor wherein the phosphor may convert energy from the LED to adifferent wavelength. An LED system is one type of illumination source.

The term “illuminate” should be understood to refer to the production ofa frequency of radiation by an illumination source. The term “color”should be understood to refer to any frequency of radiation within aspectrum; that is, a “color,” as used herein, should be understood toencompass a frequency or combination of frequencies not only of thevisible spectrum, including white light, but also frequencies in theinfrared and ultraviolet areas of the spectrum, and in other areas ofthe electromagnetic spectrum.

FIG. 5 illustrates a block diagram of one embodiment of an illuminationsystem 100. A processor 2 is associated with several controllers 3. Thecontrollers 3 control the power to the LEDs 4. As used herein, the termprocessor may refer to any system for processing electronic signals. Aprocessor may include a microprocessor, microcontroller, programmabledigital signal processor, other programmable device, a controller,addressable controller, microprocessor, microcontroller, addressablemicroprocessor, computer, programmable processor, programmablecontroller, dedicated processor, dedicated controller, integratedcircuit, control circuit or other processor. A processor may also, orinstead, include an application specific integrated circuit, aprogrammable gate array, programmable array logic, a programmable logicdevice, a digital signal processor, an analog-to-digital converter, adigital-to-analog converter, or any other device that may be configuredto process electronic signals. In addition, a processor may includediscrete circuitry such as passive or active analog components includingresistors, capacitors, inductors, transistors, operational amplifiers,and so forth, as well as discrete digital components such as logiccomponents, shift registers, latches, or any other separately packagedchip or other component for realizing a digital function. Anycombination of the above circuits and components, whether packageddiscretely, as a chip, as a chipset, or as a die, may be suitablyadapted to use as a processor as described herein. It will further beappreciated that the term processor may apply to an integrated system,such as a personal computer, network server, or other system that mayoperate autonomously or in response to commands to process electronicsignals such as those described herein. Where a processor includes aprogrammable device such as the microprocessor or microcontrollermentioned above, the processor may further include computer executablecode that controls operation of the programmable device. In anembodiment, the processor 2 is a Microchip PIC processor 12C672 and theLEDs 4 are red, green and blue.

The controller 3 may be a pulse width modulator, pulse amplitudemodulator, pulse displacement modulator, resistor ladder, currentsource, voltage source, voltage ladder, switch, transistor, voltagecontroller, or other controller. The controller controls the current,voltage or power through the LED 4. The controller also has a signalinput wherein the controller is responsive to a signal received by thesignal input. The signal input is associated with the processor suchthat the processor communicates signals to the signal input and thecontroller regulates the current, voltage and or power through the LED.In an embodiment, several LEDs with different spectral output may beused. Each of these colors may be driven through separate controllers.The processor and controller may be incorporated into one device. Thisdevice may power capabilities to drive several LEDs in a string or itmay only be able to support one or a few LEDs directly. The processorand controller may also be separate devices. By controlling the LEDsindependently, color mixing can be achieved for the creation of lightingeffects. In an embodiment, memory 6 may also be provided. The memory 6is capable of storing algorithms, tables, or values associated with thecontrol signals. The memory 6 may store programs for controlling theLEDs 4. The memory may be memory, read-only memory, programmable memory,programmable read-only memory, electronically erasable programmableread-only memory, random access memory, dynamic random access memory,double data rate random access memory, Rambus direct random accessmemory, flash memory, or any other volatile or non-volatile memory forstoring program instructions, program data, address information, andprogram output or other intermediate or final results. A program, forexample, may store control signals to operate several different coloredLEDs 4. A user interface 1 may also be associated with the processor 2.The user interface may be used to select a program from memory, modify aprogram from memory, modify a program parameter from memory, select anexternal signal or provide other user interface solutions. Severalmethods of color mixing and pulse width modulation control are disclosedin U.S. Pat. No. 6,016,038 “Multicolored LED Lighting Method andApparatus,” the entire disclosure of which is incorporated by referenceherein. The processor 2 can also be addressable to receive programmingsignals addressed to it.

There have been significant advances in the control of LEDs. U.S.Patents in the field of LED control include U.S. Pat. Nos. 6,016,038,6,150,774, and 6,166,496. U.S. patent application Ser. No. 09/716,819for “Systems and Methods for Generating and Modulating IlluminationConditions” also describes, among other things, systems and controls.The entire disclosure of all these documents is herein incorporated byreference. In embodiments of the invention, the lighting system may beused to illuminate a real world environment. On such environment isshown in FIG. 1. In FIG. 1 a user (101) is using a computing device(103). The computing device has a visual display (104). The user alsohas at least one lighting fixture (105) mounted nearby. Generally thelighting fixtures (105) will be mounted in a manner that the user (101)can see either the illumination projected by a lighting fixture (105)directly, or indirectly, such as by bouncing the illumination off asurface (107). The lighting fixtures in combination comprise thelighting system. The lighting system may be in communication with thecomputing device (103) by any manner known to one of skill in the artwhich can include, but is not limited to: wired connections, cableconnections, infrared (IR) connections, radio frequency (RF)connections, any other type of connection, or any combination of theabove. In an embodiment, control may be passed to the lighting systemvia a video-to-DMX device, which provides a simple way of generating thelighting signal. Such a device may have a video-in port and apass-through video-out port. The device may also have a lighting signalport where the DMX, or other protocol data, is communicated to thelights in the room. The device may apply an algorithm to the receivedvideo signal (e.g. average, average of a given section or time period,max, min) and then generate a lighting signal corresponding to thealgorithm output. For example, the device may average the signal overthe period of one second with a resultant value equal to blue light. Thedevice may then generate blue light signals and communicate them to thelighting system. In an embodiment, a simple system would communicate thesame averaged signal to all of the lights in the room, but a variantwould be to communicate the average of a portion of the signal to oneportion of the room. There are many ways of partitioning the videosignal, and algorithms could be applied to the various sections of thelight system, thus providing different inputs based on the same videosignal.

In the depicted embodiment the surface (107) comprises a screen designedto reflect light. Alternatively the surface could be, but is not limitedto, a wall or other surface upon which light could be reflected. Inanother embodiment, the surface could be designed to absorb andretransmit light, possibly at a different frequency. For instance thesurface (107) could be a screen coated with a phosphor whereillumination of a particular color could be projected on the screen andthe screen could convert the color of the illumination and provide adifferent color of illumination to the user (101). For instance theprojected illumination could primarily be in the blue, violet orultraviolet range while the transmitted light is more of a white.

In an embodiment, the lighting system is placed in a real worldenvironment (109) that includes the computing device (103) and the user(101). The real world environment (109) could be a room that includes acomputer. The lighting system could be arranged, for example, to lightthe walls, ceiling, floor or other sections or objects in a room insteadof, or in addition to lighting the surface (107). The lighting systemmay include several addressable lighting systems with individualaddresses. With this system, much like the surface (107), theillumination can be projected so as to be visible to the user (101)either directly or indirectly. That is a lighting fixture (105) couldshine so that the light is projected to the user without reflection, orcould be reflected, refracted, absorbed and reemitted, or in any othermanner indirectly presented to the user (101).

Referring to FIG. 7, a flow diagram 700 depicts basic steps forproviding for coordinated illumination of an environment in conjunctionwith execution of a computer application such as a game. At a step 702,the provider of the system provides for obtaining a content signal 702that relates to content from the computer game. For example, the contentsignal may be computer code for execution of the computer game, or avideo or other signal that comes from the computer game system fordisplay on a television or a monitor. At a step 704 the host establishesa system for controlling illumination of a real world environment, suchas installing lights in a desired configuration, such as an array ofcolor LEDs located in the area of the user and positioned to illuminatethe surface 107 of FIG. 1. The system further includes a processor orother element for allowing the host to change illumination. Next, at astep 708, the user coordinates the illumination control with the natureof the content signal obtained at the step 702. For example, uponreceiving certain code or a certain signal from the computer gamesystem, the illumination system may be controlled to change theillumination in the environment. Further detail will be provided inconnection with the embodiments described below.

The surface (107) may also include one or more colors, figures, lines,designs, figures, pictures, photographs, textures, shapes or othervisual or graphical elements that can be illuminated by the lightingsystem. The elements on the surface can be created by textures,materials, coatings, painting, dyes, pigments, coverings, fabrics, orother methods or mechanisms for rendering graphical or visual effects.In embodiments, changing in the illumination from the lighting systemmay create visual effects. For example, a picture on the surface (107)may fade or disappear, or become more apparent or reappear, based on thecolor of the light from the lighting system that is rendered on thesurface (107). Thus, effects can be created on the surface (107) notonly by shining light on a plain surface, but also through theinteraction of light with the visual or graphical elements on thesurface.

In use, the illumination system can be used to provide information tothe user (101) in response to or in coordination with the informationbeing provided to the user (101) by the video display (104). One exampleof how this can be provided is in conjunction with the user playing acomputer game on the computing device (103). The light system may beused to create one or more light effects in response to action on thevideo display (104). The lighting effects, or illumination effects, canproduce a vast variety of effects including color-changing effects;stroboscopic effects; flashing effects; coordinated lighting effects;lighting effects coordinated with other media such as video or audio;color wash where the color changes in hue, saturation or intensity overa period of time; creating an ambient color; color fading; effects thatsimulate movement such as a color chasing rainbow, a flare streakingacross a room, a sun rising, a plume from an explosion, other movingeffects; and many other effects. The effects that can be generated arenearly limitless. Light and color continually surround the user, andcontrolling or changing the illumination or color in a space can changeemotions, create atmosphere, provide enhancement of a material orobject, or create other pleasing and or useful effects.

It is important to note that the lighting system producing illuminationin response to the information provided on the screen allows thelighting system to describe or indicate activity that is not representedon the screen in any way. The best example of this is by thinking of theuser (101) using the video display (104) as a viewport into a virtualenvironment created by the computing device (103). In the virtualenvironment, something could be behind the user, however, the videodisplay may only show a view in front of the user. The user could “turnaround” and see the object behind them by rotating the viewport. Objectswhich are behind the user in this virtual environment could still beobjects which a user in a real environment would be able to see becausetheir vision is not limited by a viewport into the real environment. Theview through the viewport, however defines a particular positioningwithin the virtual world (a placement and a facing). Therefore, thelighting system could show or indicate information about what is behindthe user. This information is still produced in coordination with whatis produced on the video display because the particular video displaydefines the user's location in the virtual world. The lighting systemtherefore provides information in coordination with or in response tothe video display that is not actually pictured on the video display. Inparticular, it provides information that there is something notcurrently in view of the viewport of the visual display, but present inthe virtual world.

In one embodiment, the lighting system may be used to extend the area ofthe user's (101) vision beyond the edges of the video display. Referringto FIG. 1, the user could be playing a game where the user is at thecontrols of a starship and there are various objects in a surroundingvirtual space. The video display could show the view out of the supposedfront of the starship showing other starships, planets, or other cosmicphenomena visible out the front of the starship. In many games, however,the world is not defined solely by what is in front of the starship butthe user is supposed to have a first person view into a world thatsurrounds them. For instance another starship could be behind thestarship the user is piloting in the game. The illumination system couldprovide indications as to locations of other objects beyond the visualdisplay. For example, a starship may be displayed on the video displayand the starship may appear to be irradiated with a blue light. The bluelight may be coming from a light source within the game environment,possibly a nebula that is outside the viewing parameters of the screenof the video display. The nebula may be outside the upper right handcorner of the screen for instance. The lighting system could be used tocreate a light source visible to the user at the appropriate locationbased on the apparent irradiation on the ship. In FIG. 1 this may beaccomplished by having a blue light reflect off surface (107) at thepoint appropriate for the nebula.

The illumination can be used to extend beyond the boundaries of thevideo display or the user's actual vision. For instance, in the gameworld the user's starship may be being attacked from behind. Theillumination system could pass on this information to the user byflashing lights behind the user with a red color indicating to the userthat they are being attacked from the rear. The user sees theillumination as it passes over the user and reflects off of the surfaceor other objects surrounding the user. In this case, while there may beno indication of the rear attack on the video display, the lightingsystem provides the information to the user.

The example of a rear attack shows that a surface (107) may be replacedor augmented by light in the entire user's environment, or by othersurfaces. The entire room or environment in which the user is playingcould be used to extend the effective viewing space of a computermonitor. For example, a flash of light representing an explosion couldbe generated behind a user to create the feeling that the explosionactually occurred behind the user. This could also be coordinated withsound such that the sound and light appear to be coming from the samearea of the room. Such activity can allow for a more in-depth gameplaying environment. As discussed above the user in the game may besupposed to have a “first person” view into the game world. Bysurrounding the user with illumination, the user may be able to havereal world space provide a simulation of that world outside the displayon the video display. Further a moving object could appear to leave thevideo display and continue moving. For instance, the light from anexplosion could begin on the video display, and appear to race towardand even through the user.

As discussed above, the lighting system may make use of a surface orscreen to provide a location of the user's viewing of the projectedillumination. A screen may be useful because it can provide a set realworld environment designed to allow easy control over the illuminationoutside the video display. In embodiments of this invention, there is ascreen unit that is associated with the computer and that is used toreflect color output from the light system. In an embodiment, the screenunit is an enclosure such as a cabana that is used to surround thedisplay screen of a computer, wherein the cabana reflects back colorfrom color lights. One embodiment of such a cabana is shown in FIG. 2.The cabana in FIG. 2 includes a frame 201 supporting fabric 203 in arigid or semi rigid structure such as a miniature tent. The fabric couldbe selected to provide a high reflectivity and good visualization oflight. Further, the cabana having a quarter spherical shape, or otherthree-dimensional shape, can allow distance to be portrayed without useof imposed perspective. Such a design may also allow for the placementof the surface over a standard video display. The cabana in FIG. 2 alsoincludes a mounting bar (205) upon which lighting fixtures may bemounted either temporarily or permanently. The advantage of such amounting bar is that the cabana could be assembled and disassembled andthe lighting fixtures could be very quickly aimed to the appropriatepoints on the fabric (203) even if the assembler did not know theappropriate aiming of the lights. In one embodiment, clips or othermounting devices can be included on mounting bar (205) that only allow aparticular installation of a particular fixture aimed in a particulardirection. Other examples of screens and enclosures for reflecting lightfrom the light system are set forth in appendix A. The enclosures couldbe anything from small, collapsible units, to large, evenbuilding-sized, arrangements. Any enclosure designed to be associatedwith a display screen for a computer game and to reflect color lightfrom a light system associated with the computer game should beunderstood to be encompassed by the present disclosure. In an embodimentof the invention, there may be a light screen permanently associatedwith a computer for reflecting light that represents an attribute of anobject of a program running on the computer. The screen could be acompletely or partially enclose an environment, such as computersurrounded by an enclosure of white or other material. One example couldbe a completely enclosing arcade game setup as is known to the art. Anynumber of lights (lighting fixtures) may be used to light the screen.The one or more lights may include several addressable lighting systemsand the projected illumination from individual lighting fixtures may bedistinct, overlapped, or both on the screen. The screen itself couldalso contain lighting fixtures not aimed at the screen; for instance,lighting fixtures could be placed on mounting bar (205) and aimed at theuser.

There are many ways that the lighting fixtures can be arranged andlighting systems can be generated to illuminate a surface or shinedirectly on the user. FIG. 3 illustrates an embodiment where thelighting system (1404) is incorporated onto a housing for the videodisplay (1402). This device can be used in place of a more complicatedlayout of lighting units, or it can be used in conjunction with theother lighting units. The lighting system (1404) can be divided up intoseveral sections to provide stereo or other multi-channel lightingeffects around the video display. For example, the lighting system(1404) can be divided into three segments, right side, left side andtop. A light track could contain light control information for each ofthese sections such that light of any color can be emitted from any sideof the video display (1402). This may be useful in extending the effectsfrom the video display (1402) onto a wall or screen behind it. Forexample, if the right side of the video display is blue the lighting onthat side of the video display (1402) could also light the wall blue.Flashes of lightning that cover the entire screen could be flashed allaround the video display (1402). If an automobile's headlights start onthe right side of the screen and move to the left, the lighting system(1404) could follow the same pattern.

FIG. 4 illustrates an embodiment where the lighting system (1504) isincorporated with the speaker (1502) of a sound system. This device canbe used in place of a more complicated layout of lighting units or itcan be used in conjunction with the other lighting units. The lightingunit (1504) can also be divided up into sections although a largebenefit from this embodiment is to use it as a single lighting elementto project the light in various sections of the room where the user hadalready placed speakers. Therefore setup of a surround sound system, aswould be understood by one of skill in the art, could also allow for thesetup of a surround light system at the same time. The benefits of thisare apparent. For example, the sound of an explosion traveling over theuser via the speaker can be complemented by the light of the explosiontraveling in the same manner.

The examples discussed above primarily relate to systems where a realworld light system in the user's environment can generate light outputthat reflects, indicates, or is associated with objects or events in thevirtual environment of a computer game. Conversely, the virtualenvironment can reflect events in the real world environment, such aswhen the lights are coupled with sensors, receivers, or other inputs forreceiving data. For instance, a sensor could detect that the light froma particular fixture is being blocked, indicating the user has stood upand is taking a break from the game and the fixtures should be powereddown to save energy, and the user's game should be paused. A wide rangeof detected conditions could be used to provide input or feedback to thegame and should be understood to be encompassed herein, for exampledetection of the presence of absence of the user in room or in proximityto the display screen, biometric characteristics of the user, such asheart rate, blood pressure, body temperature or the like, room lighting,temperature, and sound levels, and many others. In embodiments thedetected conditions can be used to influence the game. For example, if auser's pulse reached a certain level, the game could alter play toprovide a less (or more) stressful set of events.

In embodiments, the light system may be associated with code for thecomputer game, so that the computer game code is modified or created tocontrol the light system. For example, object-oriented programmingtechniques can be used to attach attributes to objects in the computergame, and the attributes can be used to govern behavior of the realworld light system. Object oriented techniques are known in the field,and can be found in texts such as “Introduction to Object-OrientedProgramming” by Timothy Budd, the entire disclosure of which is hereinincorporated by reference. It should be understood that otherprogramming techniques may also be used to direct lighting systems toilluminate in coordination with games, object oriented programming beingone of a variety of programming techniques that would be understood byone of ordinary skill in the art to facilitate the methods and systemsdescribed herein.

In an embodiment, a developer can attach the light system inputs toobjects in the game. For example, the developer may have an abstractionof a light that is added to the code construction, or object, of a gameobject. An object may consist of various attributes, such as position,velocity, color, intensity, or other values. A developer can add lightas an instance in the object in the code of a game. For example, thegame object could be a ship, with attributes, such as position, size,velocity, etc. A light source can be added as an instance of the objectof the game, and the light source can have attributes, such asintensity, color, and various effects. Thus, when events occur in thegame that call on the object of the ship, a thread running through theprogram can draw code to serve as an input to the processor of the lightsystem. The light can accurately represent geometry, placement, spatiallocation, represent a value of the attribute or trait, or provideindication of other elements or objects.

Referring to FIG. 8, a flow chart 800 provides steps for a method ofproviding for coordinated illumination. At the step 802, the programmercodes a game object for a computer game, using, for example,object-oriented programming techniques. At a step 804, the programmingcreates instances for each of the objects in the game. At a step 808,the programmer adds light as an instance to one or more objects of thegame. At a step 810, the programmer provides for a thread, runningthrough the game code. At a step 812, the programmer provides for thethread to draw lighting system input code from the objects that havelight as an instance. At a step 814, the input signal drawn from thethread at the step 812 is provided to the lighting control system, sothat the lighting system responds to code drawn from the computer game.

Using such object-oriented light input to the light system from code fora computer game, various lighting effects can be associated in the realworld with the virtual world objects of a computer game. For example, ina space battle game, a ship's light source can be attached with aneffect, such as sound, flashing, motion, vibration and other temporaleffects. Further, the light system could include other effects devicesincluding sound producing devices, motion producing devices, fogmachines, rain machines or other devices which could also produceindications related to that object.

Referring to FIG. 9, a flow diagram 900 depicts steps for coordinatedillumination. In embodiments, the program code for the light has aseparate thread running on the machine. At a step 902 the programinitiates the thread. At a step 904 the thread as often as possible runsthrough the list of virtual lights. At a step 908 the thread doesthree-dimensional math to determine which real-world lights are inproximity to reference point in the real world (e.g., the head of theuser) that is projected as the reference point of the coordinate systemof objects in the game. Thus, the (0,0,0) position can be the user'shead in the real world and a point on the screen in the game (forinstance the center of the video display and therefore the user's viewinto the virtual environment). At a step 910, the code maps the virtualenvironment and object in it to the real world environment, includingthe light system, so that events happening outside the computer screenare similar in relation to the reference point as are virtual objects toa reference point on the screen. At a step 912, the host may provide aninterface for mapping. The mapping function may be done with a function,e.g., “project-all-lights,” as described in Directlight API describedbelow and in Appendix A, that maps real world lights using a simple userinterface, such as drag and drop interface. The placement of the lightsmay not be as important as the surface the lights are directed towards.It may be this surface that reflects the illumination or lights back tothe user and as a result it may be this surface that is the mostimportant for the mapping program. The mapping program may map thesesurfaces rather than the light fixture locations or it may also map boththe locations of the fixtures and the light on the surface. In oneembodiment a screen, such as the cabana discussed above, can furthersimplify this mapping by providing a set and unchanging lighting systemand screen that will have identical properties no matter the real worldenvironment in which the system is located.

A system for providing the code for coordinated illumination is depictedin FIG. 10, which may be any suitable computer capable of allowingprogramming, including a processor 1002, an operating system 1004, andmemory, such as a database 1008, for storing files for execution.

Each real light may have attributes that are stored in a configurationfile. An example of a structure for a configuration file 1100 isdepicted in FIG. 11. The configuration file 1100 may include variousdata, such as a light number 1102, a position of each light 1104, theposition or direction of light output 1108, the gamma (brightness) ofthe light 1110, an indicator number for one or more attributes1112-1114, and various other attributes. By changing the coordinates inthe configuration file, the real world lights can be mapped to thevirtual world in a way that allows them to reflect what is happening inthe virtual environment. The developer can thus create time-basedeffects, such as an explosion. There can then be a library of effects inthe code that can be attached to various game attributes. Examplesinclude explosions, fades in and out, etc. The developer attaches theeffects to virtual lights in the game. For example, when an explosion isdone, the light goes off in the game, reflecting the destruction of theobject that is associated with the light in the configuration file.

To simplify the configuration file, various techniques can be used. Inembodiments, hemispherical cameras, sequenced in turn, can be used as abaseline with scaling factors to triangulate the lights andautomatically generate a configuration file without ever having tomeasure where the lights are. Referring to the flow diagram 1200 of FIG.12, in embodiments, the configuration file is created at a step 1202.The configuration file can be typed in, or can be put into a graphicaluser interface that can be used to drag and drop light sources onto arepresentation of a room. At a step 1204, the developer can create aconfiguration file that matches the fixtures with true placementrelative to a user's coordinate in the real room. For example, once thelighting elements are dragged and dropped in the environment, at a step1208 the program can associate the virtual lights in the program withthe real lights in the environment. An example of a light authoringprogram to aid in the configuration of lighting is included in U.S.patent application Ser. No. 09/616,214 “Systems and Methods forAuthoring Lighting Sequences.” Color Kinetics Inc. also offers asuitable authoring and configuration program called “ColorPlay.”

Further details as to the implementation of the code can be found in theDirectlight API document attached hereto as Appendix A. Directlight APIis a programmer's interface that allows a programmer to incorporatelighting effects into a program. Directlight API is attached in AppendixA and the disclosure incorporated by reference herein. Object orientedprogramming is just one example of a programming technique used toincorporate lighting effects. Lighting effects could be incorporatedinto any programming language or method of programming. In objectoriented programming, the programmer is often simulating a 3D space inthe screens. The 3D space continues throughout the room the gamer is inand beyond to virtual infinity. In this 3D space, the programmer mayplace an object, such as a starship, as well as a virtual light, such asa nebula, sun, star, other star ship, photon beam, weapon, explosionplume or other light generating object.

In the above examples, lights were used to indicate the position ofobjects which produce the expected light or have light attached to them.There are many other ways in which light can be used. The lights in thelight system can be used for a variety of purposes, such as to indicateevents in the game, to indicate levels or attributes of game objects,such as characters, ships, weapons, shields, health, playing pieces,environments, rooms, or other objects. For instance the lights couldturn to a particular color (for instance green) representing the shieldstrength of a starship. As the shields were slowly knocked down by enemyfire, the lights could change color or slowly fade out indicating theshield strength is fading. The lights can be used for aestheticpurposes, such as to connote a mood for an object or an environment inthe game. For instance, the lights could change color to indicate thepassage of time in the game world, or could produce lighting effects tosimulate environment effects. For instance the lights could have anappropriate flicker to indicate that it is raining in the game world. Awide range of effects can be used, each associated with a particularobject or event in the game. The lighting system could provide furtherindications such as by triggering a real world fog machine, when thevirtual environment is supposed to be foggy.

In embodiments, there may be an be an optional override mechanism thattakes over one or more lights and direct them to do something regardlessof what the virtual light source is intended to do. This may be usefulwhen the program requires all of the lights in the system to change to aparticular color. For example, the game may be played out in a spaceenvironment where the starship is lighted blue from a nearby nebula,when suddenly the ship is attacked by another ship. The program caninstantly change all of the lights in the system to flashing red toindicate a red alert. The pulsation of the lights may be coordinatedwith the sound of an alarm or other sensations to further immerse theuser.

In embodiments, an object, such as a ship's light source, can beassociated with lights outside the game or any purposeful lightingsetup, such as room lights. Any real world lighting fixture could beused as part of a lighting system as discussed herein Optionally, directaccess to outside lights can override the light source in the game. Forexample, the video display can provide user interface information, suchas red alerts, strength of a unit or other attributes of game objectsthrough the lights. In effect the video display could be disabled andonly lights could be used. Such an embodiment could enable a computergame to be created which uses the lights as the sole source ofinformation, or alternatively, the video display device could bedisabled to indicate a particular situation has occurred. Returning tothe starship example, a situation could occur where the starship hasbeen “blinded” as part of the game actions. This may result in theforward view being unavailable to the user. However, the user can stillnavigate the starship using the lighting system. This ability can add adepth of play that has previously been unavailable to a computer gamebecause the screen video display has previously been the only thatprovides visual feedback.

It is also possible to have light system lights that are not attached toobjects in the game, such as to indicate another environmentalcondition, such as the end of the work day, sunset, sunrise, or someother indicator that is useful to a player immersed in the game. The mayalso provide mood or aesthetics such as projecting the presence of aperson, creature object, or other thing such as by an aura or theirtraits of good and evil. These traits could be associated with colorsand intensities. Approaching a dangerous object could also have thelights switch to a warning mode (such as flashing red) to warn the userof the danger.

In yet another embodiment, the lights could act as a detriment to gameplay to show various conditions. For instance, in certain computer gamesthe user is supposed to be hampered because they have carried out aparticular action in the game. For instance the sound may be cut offbecause the user was to close to a virtual explosion and they were“deafened” by it. Such feedback is common, often with the user's videodisplay acquiring random noise or color to disorient the user. Oneexample of how lights could be used if a player was too close to anexplosion, the lights could provide for blinking “spots” to distract theuser and degrade their game play. Alternatively, if the game universeindicated the user was supposed to be drunk, disoriented, or otherwisedistracted, the lights could flash or change color is disorientingpatterns to actually distract the user from their game play.

In the embodiments above, a system is primarily discussed where thecomputer code is designed to interact with a three-dimensional virtualworld of which the user is part. The code works well to translate fromthe three dimensional objects into the real three-dimensional world. Tohook up the lights to a two-dimensional world, additional programmingmay be required and additional conditions may need to be put intoeffect. In a two-dimensional world, the code is not necessarily designedto place the real world user at a reference point, so the code may needto adapt to place the real world user in the environment in the sameposition as the virtual “user” (called a character) in the game.Alternatively, it may be undesirable to control the lighting systemsillumination to indicate what the user's character is seeing in thevirtual world. Instead, the lighting system may extend the world theuser sees. One example of a 2-D game is “Diablo” produced by Blizzardsoftware. In this game the user looks down on a map which contains arepresentation of the character the user is controlling. As opposed tothe first person view discussed above, in this game the user has a thirdperson view looking down on the character that is supposed to be them.The screen then provides a perspective layout of a game world of whichthe character is a part. If the illumination system provides the view asthe character would see it, the user may be confused (for instance afire could be burning at the lower right corner of the screen but berepresented by a lighting fixture behind and to the left of the user,based on the facing of the character, the character could then turn inplace resulting in minimal change of the information on screen, but adramatic shift in the illumination produced by the lighting system. Itmay therefore be desirable to extend any perspective or view of a 2-Dgame in a manner so that the lighting system illuminates in a manner toconform with the view on the video display. Following with the aboveexample the lights may track to follow the fire off screen as thecharacter moves away from the fire. This could occur by defining a“plane of the game” representing the game world as if the user was notpresent at all and only the character existed. Therefore as thecharacter in the game moved up and left from the fire, the fire appearsto move, via the lighting system, down and to the right.

In one embodiment of the invention, the lighting system can be replacedor augmented by lights already present in the real world environment.For instance, one can create a game that involves the use of the lightsin the house. One can use the game itself as a user interface for thelights in the house. A good example might be a horror game where whenthe lights go out in the game environment, they also go out in the realworld room. Such an environment could be highly engaging to a user whois placed even more within the virtual world in which they are playingby having that world truly interact with the real world where they are.

The real lights in the house could be made game objects for the gameitself, or particular light arrangements could be created for thepurposes of playing particular games. Referring to FIG. 13, a system1300 for using an array of lights 1302 in conjunction with a computersystem 1304 (which can be any conventional computer system that providesa connection 1308 to the lights 1302. The array 1302 can be disposed ona wall or the ceiling, as depicted in FIG. 13. The individual LEDs orother lights in the array 1302 can be provided with changes in on-offstatus, color, and intensity, serving in effect like pixels on display.Thus, a version of a game such as “pong” or other simple game can beplayed with an array of lights in the light system, with the lights inthe array serving as “pixels” analogous to the virtual environment onthe screen. In such embodiments it may be unnecessary to use the screenof the computer system 1304 to play the computer game. Alternatively,one player could use the screen to control the game being played, whileother players are totally immersed in a gaming environment. As discussedabove, a player could be playing a horror game while the players in thenext room are experiencing the light sensations from his play withoutcontrol over the game environment. Essentially they can experience thegame play as a third party along for the ride in the game. Further thelayout of lights in a house could denote the parameters for a particulargame. For instance, a game could be created where the user is fightingoff evil monsters in their own home and the lighting in their home isacting in the same manner as the lighting in the game. Such a gameenvironment could be creating by receiving information about thelocation of lights in a house, and generating the game world to conformto that lighting.

FIG. 14 depicts a flow chart 1400 with steps for programming a system tocoordinate house lights with a game. At a step 1402 the programmerlocates the real-world location of each of the house lights. At a step1404 the programmer maps the lights, such as by dragging and droppingrepresentations of the locations of the lights onto a virtualrepresentation of the house on the display screen of the computer. At astep 1408 the program generates a game world that contains virtuallights that match the lights of the real world. Then, changes in thegame lights can change the room lights, such as in hue or intensity.

It would be easily understood by one of skill in the art that thismulti-party experience could be extended to multi-party competitivegames such as those commonly played across networks. It could be thecase where a user is actually trying to affect the real worldenvironment in which another user is playing. For instance, in the gamea first user could be trying to disable an opponent's ship by knockingout its ability to perceive the world around it. For instance it couldknock out the ship's forward or rear views. In the real world a secondplayer could actually have various views knocked out (lights beingdisabled) as the first player accomplishes his goal. In an even morerealistic example, the first user could be trying to turn the lights outon the second user. The first person may also directly affect the otherperson's room lights by, for example, turning the lights off or changingtheir color, or the first person may indirectly change the lightingconditions in the other persons room by, for example, getting close tothe other persons virtual room and tripping a sensor. The first personmay also be carrying an object that generates light or reflects light.This object may trigger the lighting in the other person's room toindicate or warn the other person of the first persons presence.

An embodiment may also be an indicator light. A problem with gamingsystems and other computer systems in general is the lack of space on avideo display. There is only a limited number or amount of dials,indicators, or other status checks that can be placed while stillproviding a user with a reasonable play environment. A gaming program,for example, may generate an indicator on the video display to indicatea parameter such as the health of the ship, the fuel remaining, thestrength of the shields or other parameters. This information isimportant to the user; however, the indicator takes up valuable videoscreen, leaving the actual game play area a diminished space andpossibly cutting off important user views. An embodiment of theinvention may be an indicator light or set of lights that sits on top ofthe computer screen, for example.

FIG. 6 depicts a computer screen 602 which could be useful to providesuch indicators. An information system 601 may be associated with thecomputer screen 602. The information system 601 may contain lightingelements 603. In an embodiment, the lighting elements 603 may be alignedas separate lights that can turn on and off and or change in hue,saturation or intensity. For example, the information system may haveseveral independently controllable LED systems and these systems couldbe controlled in such a manner as to change color to indicateinformation. For instance, the information could be communicated to theinformation system through gaming software to indicate the health of anobject or person, indicate shield strength, indicate fuel level or anyother information that may be useful for the user. In an embodiment, theinformation system may take the form of a single light source indicatingone or more parameters.

The indicators could perform a wide variety of functions includingturning on and off, changing color, changing intensity and or otherfunctions to replace or additively improve upon the output from theindicators. For instance, a user's health in the game could be displayedby a number in the corner of the user's screen but could also berepresented by a light that changes from green to red as the virtualhealth drops and flashes insistently when the user's virtual health isperilously low.

In another embodiment, a shield could start as white and fade to anothercolor as it loses strength. The user could start to see “holes” in ashield that are shown by different colors in the room in the directionwhere the shield is weak allowing a user to quickly realize how to turnto bring more powerful shields to face an enemy. You could see also seea jump into “warp” speed where the front of the room turns bright andthen the bright light flashes towards the back of the room, for example.In a racing game, speed could also be simulated as streetlights onscreen race down a row of lights off to one side of the user.

In an embodiment, a lit enclosure for the device on which the game isbeing played changes as an indicator. Game console buttons could alsoblink to indicate a combination. Like a color-note organ (such asdescribed below), this embodiment could be used in training mode toteach combinations.

Although the figures and description above show primarily computergames, it would be clear to one of skill in the art to carry the systeminto other types of computing devices and environments. A computingdevice can include, but is not limited to, any type of computer, aconsole game system (such as the Playstation series manufactured bySony), a Personal Digital Assistant (PDA), a telephone, a WebTV orsimilar system, a thin client, or any other type of system on which auser is able to carry out applications where a lighting system couldenhance the display provided to the user. There can be systems where thelighting system provides the only source of visual information to theuser.

For console game systems, one of skill in the art would understand thatlibraries customized onto the proprietary chipsets for console gamesthat drive light system output, similar to the Directlight programmer'sinterface, could be created without undue experimentation. Console gamesgenerally have proprietary chipsets so it may be necessary to generatecustom libraries for these systems. The systems and libraries for theconsoles could function in much the same way as a PC-based game. Theconsole may include a USB, serial, parallel, firewire, optical, modem orother communications port to communicate with the lighting system. Thelighting information could also be sent through a controller port. Acontroller port may be used for a controller communication as well aslighting control information. Separate controller ports could also beused. For example, a first controller port may be used to communicatewith a controller and another controller port may be used to communicatewith the lighting system.

Many games and computer systems include input devices such as ajoystick, mouse, keyboards, gloves, tactile mouse, dance pads, exerciseequipment, or other input devices. These devices are generally used by auser to control aspects of a game or other parameter of a computerprogram. Each of these input devices could also be configured to affectthe room lights. For example, a mouse could be used to control thelights in a room. As the mouse is moved the lights in the room couldrespond, or as the user dances on a dance pad the lights could generatea color representation of their dance. For instance, their impact forcewith the pad could translate to an intensity while their positiontranslated to a color. The input device may also direct soundsimultaneously or in conjunction with the light.

In an embodiment of the invention the lighting system could also beassociated with room inputs that could be associated with the virtualenvironment such as a microphone, camera, heat, cooling or other roominputs. For example, a user could control a game object by providingvoice instructions through the microphone, which could be synthesizedinto commands for an application, and in turn used to control theillumination of the environment through a lighting system.

The embodiments discussed above relate primarily to games involvingreal-time simulation and for such types of games there are numerousapplications for lighting systems. For instance: flight games could useindicators for controls or important statistics like fuel level; racinggames could have motion or indicate third party activities like theapproach of police vehicles: skateboarding, snowboarding, or otherperformance sport simulators can have indicators of movement, indicatorsof third party actions, or rewards such as flashbulbs for particularlyfine performances. Other types of simulators can use lighting systemsincluding, but not limited to, roller-coaster simulations, closed bootharcade simulators, or location-based entertainment games (large gamesinside a booth with multiple players). Further, it would be understoodby one of skill in the art that the above are merely a limited overviewof possibilities and there are many more applications that could beperformed without undue experimentation.

Simulation types of games are typically 3D rendered and have objectswith attributes as well as events. Referring to the flow chart 1500 ofFIG. 15, at a step 1502 a programmer can code events into the game. At astep 1504 a programmer can also code attributes or objects. Thus, a gameprogram can track events and attributes, such as explosions, bullets,health, other people, patterns of light, and/or secret passageways. At astep 1508, the code can then map from the virtual world to the realworld. In embodiments, at an optional step 1510, the system can add tothe virtual world with real world data, such as from sensors or inputdevices. Then, at a step 1512, the system can control real and virtualworld objects in coordination with each other. Also, by using the lightsystem as an indicator, it is possible to give information through thelight system that aids the user in playing a game or otherwiseinteracting with a virtual environment. For example, a light could flashor change color to indicate the coming of an event that would otherwisebe a surprise to a user who does not have the light system when playingthe game. The indicator functions of the light system can be coupledwith three dimensional sound (such as surround sound) to provideenhanced real world effects that relate to events and attributes in thevirtual environment.

In other embodiments the lighting systems can be used in conjunctionwith puzzle games, a developer can create a room-puzzle using the lightsystem in a room or other environment to create games with the lights.For example, the room lights can reflect the color of puzzle elements inthe virtual world, creating a correspondence between the virtual and thereal world puzzle elements. One can manipulate real-world elementswithout the display on the computer. (In embodiments, one can play agame, such as pong, with the actual lights using a joystick attached tothe computer. The code for the game can provide the interactive aspectsof the game, then hand control signals to the light system so that thelights act as game elements in the real world environment. For example,one can create an array of lights on the ceiling, which could send lightelements across the room). The lights can act as “pixels” in an array onthe ceiling, wall or floor. In embodiments, the arrays could beestablished in a variety of rooms, so that light output moves from roomto room in response to the user's interaction with the input device orsensor of the computer game or other virtual environment. Light stripscan be used for game elements, as can other light system configurations.

A real world light system can be used to respond to input from objectsin classic arcade computer games, such as Pacman, pong, asteroids, spaceinvaders, breakout, and similar games, in each case enhancing the userexperience by providing either aesthetic or indicator functions. Lightsystems can also be used with any other game type, such as casinosimulators, video poker, sandbox games, railroad tycoon games,simulation games like Sim City, and the like.

In embodiments, light systems can be used with role-playing games, suchas two-dimensional games like Diablo and three-dimensional games likeEverQuest (a massively multi-player online game). In such games, realworld lights can be mapped to attributes of humans or objects in realworld. The lights can provide more interaction with other humans ornon-human players. Attributes and events can be a wide variety ofthings, such as indicating health, casting a spell, indicating shieldstrength or presence, displaying a fireball, or other effects. Further,in multiplayer games at a single location, the lights can be used toprovide functions to the players by indicating the seat of a player youwould like to meet in person, for example.

Further, in many multi player games (such as Everquest) items andcharacters have become collectible. Everquest Avatars are popular wherethere are characters that have characteristics that are acquired overtime. Some are sold for large amounts of money on the Internet. The roomlights of a light system can reflect the attributes of an avatar, suchas a powerful blue to represent a “good” character, or a dark red toreflect “evil.” Any attribute, ranging from personality traits, tostrength, speed, location, health, constitution, intelligence, stamina,wisdom, or other attributes can be reflected for a given avatar by theroom lights in the real world in correspondence to the attributes of theavatar in the virtual world. In embodiments, one can make collectiblepatterns or effects that associate with characters, objects or events.Thus, a virtual world object or character could have a “signature” or“trademark” color effect that takes place each time the characterappears in the virtual world or does a particular thing in the virtualworld. You could accrue an aura for a character over time as part of acharacter's attributes. Light could be used to portray abstractcharacter traits. In multi-player environments like arcades, peoplecould glow according to the aura of the character they are playing.Objects could embody those traits, and you might need to have aparticular object or trait to allow you to see the indicator for anotherobject.

In adventure games, which typically involve solving a puzzle in anenvironment—environmental lighting could be part of the puzzle. Thelight system could give clues required or helpful to solve the puzzle.

In sports games, a light system in the real world can serve as an eventindicator (flashbulbs, cut scenes—could shift lights) or as an attributeindicator (indicating health of a player, number of fouls, etc.).

In turn-based strategy games, a light system can indicate events orattributes, or provide aesthetic effects. Those could include the dayand night cycle, something catching fire (where the user sees the eventfirst in the lights, then in virtual world), a warning from behind thatcomes first (such as to indicate a glowing light sabre of a playerbehind the user in the virtual world), or other events or attributes.

In fighting games, lights in the real world can indicate seeingsomeone's shadow before they jump over the user, moving lights after a“kill” or powerful blow, or indicating declining or improving health orpresence of special attributes (such as shields or spells). Further,moving shadows or illumination created by the lights could indicate theposition of a tag team partner. It is important to note that the absenceof light (a shadow) can also be part of the illumination andillumination as described herein. The absence of any intensity of anycolor is simply ones possible illumination condition.

In children's games, room lights and a room sound system can reflectcolor and intensity attributes of elements of the games, which caninclude games to teach children about different colors and sounds.

As was discussed above, computer games provide merely one type ofcomputer program in which a lighting system can provide enhancement andadditional information. There are many other types of computer programswhich the use of a lighting system can benefit

A similar embodiment to computer games is auction-type, real “games,”such as Internet auctions, such as E-bay. Lights could be used as anindicator of an event or characteristic, such as having the highestcurrent bid, or a winning bid. Online gambling establishments could alsouse the lights so as to provide a casino-like environment in the user'shome.

Architectural visualization, mechanical engineering models, and othersolid modeling environments are encompassed herein as embodiments. Inthese virtual environments lighting is often relevant both in a virtualenvironment and in a solid model real world visualization environment.Referring to FIG. 16, a system 1600 includes a computer 1602 with adisplay 1604. The system 1600 also includes a solid model 1608, as wellas a lighting system 1610, with one or more lights 1612. The lightingsystem 1610 lights the solid model 1608 in the real world. Arepresentation 1614 of the solid model can be created on the displayscreen, typically a three-dimensional representation created by a solidmodeling program, such as those provided by SolidWorks. In therepresentation 1614 can be located virtual lights 1618, which can berepresented as appearing in the modeling world in directionscorresponding to the directions of the lights 1612 of the lightingsystem 1610. The user can thus position and control the light system toilluminate the real world solid model in correspondence to illuminationconditions that are created in the virtual world environment. Scalephysical models in a room of lights can be modeled for lighting duringthe course of a day or year or during different seasons for example,possibly to detect previously unknown interaction with the light andvarious building surfaces. Another example would be to construct areplica of a city or portion of a city in a room with a lighting systemsuch as those discussed above. The model could then be analyzed forcolor changes over a period of time, shadowing, or other lightingeffects. In an embodiment, this technique could be used for landscapedesign. In an embodiment, the lighting system is used to model theinterior space of a room, building, or other piece of architecture. Forexample, an interior designer may want to project the colors of theroom, or fabric or objects in the room with colors representing varioustimes of the day, year, or season. In an embodiment, a lighting systemis used in a store near a paint section to allow for simulation oflighting conditions on paint chips for visualization of paint colorsunder various conditions. These types of real world modelingapplications can enable detection of potential design flaws, such asreflective buildings reflecting sunlight in the eyes of drivers duringcertain times of the year. Further, the three-dimensional visualizationmay allow for more rapid recognition of the aesthetics of the design byhuman beings, than by more complex computer modeling.

Solid modeling programs can have virtual lights. One can light a modelin the virtual environment while simultaneously lighting a real worldmodel the same way. For example, one can model environmental conditionsof the model and recreate them in the real world modeling environmentoutside the virtual environment. For example, one can model a house orother building and show how it would appear in any daylight environment.A hobbyist could also model lighting for a model train set (for instancebased on pictures of an actual train) and translate that lighting intothe illumination for the room wherein the model train exists. Thereforethe model train may not only be a physical representation of an actualtrain, but may even appear as that train appeared at a particular time.A civil engineering project could also be assembled as a model and thena lighting system according to the principles of the invention could beused to simulate the lighting conditions over the period of the day.This simulation could be used to generate lighting conditions, shadows,color effects or other effects. This technique could also be used inFilm/Theatrical modeling or could be used to generate special effects infilmmaking. Such a system could also be used by a homeowner, forinstance by selecting what they want their dwelling to look like fromthe outside and having lights be selected to produce that look. This isa possibility for safety when the owner is away. Alternatively, thesystem could work in reverse where the owner turns on the lights intheir house and a computer provides the appearance of the house fromvarious different directions and distances.

Although the above examples discuss modeling for architecture, one ofskill in the art would understand that any device, object, or structurewhere the effect of light on that device, object, or structure can betreated similarly.

Medical or other job simulation could also be performed. A lightingsystem according to the principles of the present invention may be usedto simulate the lighting conditions during a medical procedure. This mayinvolve creating an operating room setting or other environment such asan auto accident at night, with specific lighting conditions. Forexample, the lighting on highways is generally high-pressure sodiumlamps which produce nearly monochromatic yellow light and as a resultobjects and fluids may appear to be a non-normal color. Parking lotsgenerally use metal halide lighting systems and produce a broad spectrumlight that has spectral gaps. Any of these environments could besimulated using a system according to the principles of the invention.These simulators could be used to train emergency personnel how to reactin situations lit in different ways. They could also be used to simulateconditions under which any job would need to be performed. For instance,the light that will be experienced by an astronaut repairing an orbitingsatellite can be simulated on earth in a simulation chamber.

Lights can also be used to simulate travel in otherwise inaccessibleareas such as the light that would be received traveling through spaceor viewing astronomical phenomena, or lights could be used as a threedimensional projection of an otherwise unviewable object. For instance,a lighting system attached to a computing device could provide a threedimensional view from the inside of a molecular model. Temporal Functionor other mathematical concepts could also be visualized.

Optionally, the virtual environment can be generated by a computerapplication that is a screen saver, which could be mapped into the realworld lights in a room. As the screen saver creates and displaysgraphics on a computer screen, for example, the screen saver can createlighting effects outside of the computer screen. This could be used tocreate decorative effects or the lighting effects could be used toprovide information or other effects. The information may representcomputer, or network activity, for example, such that the activity isdisplayed in the lights and or the screen saver. This could includeemail activity, when an email is received by the system the lightingcould change to a particular hue, saturation or intensity. The hue,saturation or intensity may change as more and more email is received.

A musical application could also be used, allowing for thechoreographing of music to light, or the generation of light as aportion of the generation of music. Alternatively light could be used tohelp a user learn to play music. For instance light could be projectedthat indicates a particular key a user should press on a keyboard. Intime, a user unable to read music could teach themselves to playinstruments and music for the user's performance could be provided aslight signals.

An embodiment of the present invention could be a puzzle that consistsof getting the room lights into a particular configuration. You couldput a person “inside” a real world Rubik's cube associated with avirtual Rubik's cube. An embodiment of the invention may be used inflight simulators to change the ambient lighting conditions from day tonight, or changing the lighting conditions as the horizon changes orassociated with other aspects of the simulator.

A system according to the principles of the invention may also involvereal-time simulation such as an actual motion inducing flight simulator,3D motion rides (control through 3D libraries), or full mockupsimulators where the lighting can be directly changed in response towhat occurs. One example is that in a research submarine trainer thelighting within the submarine could be altered to remove red wavelengthsas the sub dives because under water red light is often not present.

All articles, patents, and other references set forth above are herebyincorporated by reference. While the invention has been disclosed inconnection with embodiments shown and described in detail, variousequivalents, modifications, and improvements will be apparent to one ofordinary skill in the art from the above description.

1. In a system including a video display facility and a lightingfacility, a method of providing lighting effects with a video display,comprising: automatically coordinating lighting effects generated by thelighting facility with a video signal provided to the video displayfacility.
 2. The method of claim 1 wherein the lighting facility isadapted to generate colors within a color spectrum.
 3. The method ofclaim 1 wherein the lighting facility comprises an LED.
 4. The method ofclaim 3 wherein the lighting facility comprises a color-controllable LEDlighting facility.
 5. The method of claim 1 wherein the lightingfacility is integrated into the video display facility.
 6. The method ofclaim 5 wherein the integrated lighting facility is positioned toproject lighting effects behind the video display facility.
 7. Themethod of claim 6 wherein the projected light is directed to illuminatea wall.
 8. The method of claim 1 wherein the lighting facility isintegrated into an audio facility.
 9. The method of claim 8 wherein theaudio facility includes at least one speaker.
 10. The method of claim 8wherein the integrated lighting facility is positioned to projectlighting effects behind the audio facility.
 11. The method of claim 10wherein the projected light is directed to illuminate a wall.
 12. Themethod of claim 1 wherein the video display facility comprises atelevision.
 13. The method of claim 1 wherein the coordination isaccomplished at least in part through interpreting the video signal. 14.The method of claim 1 wherein the coordination is accomplished throughthe use of a combined signal.
 15. The method of claim 15 wherein thecombined signal includes lighting control information and videoinformation.
 16. In a system including a video display facility and alighting facility, a method of providing lighting effects with a videodisplay, comprising: automatically coordinating lighting effectsgenerated by the lighting facility with the video display facility. 17.The method of claim 16 wherein the lighting facility is adapted togenerate colors within a color spectrum.
 18. The method of claim 16wherein the lighting facility comprises an LED.
 19. The method of claim18 wherein the lighting facility comprises a color-controllable LEDlighting facility.
 20. The method of claim 16 wherein the lightingfacility is integrated into the video display facility.
 21. The methodof claim 20 wherein the integrated lighting facility is positioned toproject lighting effects behind the video display facility.
 22. Themethod of claim 21 wherein the projected light is directed to illuminatea wall.
 23. The method of claim 16 wherein the lighting facility isintegrated into an audio facility.
 24. The method of claim 23 whereinthe audio facility includes at least one speaker.
 25. The method ofclaim 23 wherein the integrated lighting facility is positioned toproject lighting effects behind the audio facility.
 26. The method ofclaim 25 wherein the projected light is directed to illuminate a wall.27. The method of claim 16 wherein the video display facility comprisesa television.
 28. The method of claim 16 wherein the coordination isaccomplished at least in part through interpreting a video signalsupplied to the video display facility.
 29. The method of claim 16wherein the coordination is accomplished through the use of a combinedsignal.
 30. The method of claim 29 wherein the combined signal includeslighting control information and video information.
 31. An entertainmentfacility, comprising: a video display facility and a lighting facility;and a coordination facility adapted to automatically coordinate lightingeffects generated by the lighting facility with a video signal providedto the video display facility.
 32. The facility of claim 31 wherein thelighting facility is adapted to generate colors within a color spectrum.33. The facility of claim 31 wherein the lighting facility comprises anLED.
 34. The facility of claim 33 wherein the lighting facilitycomprises a color-controllable LED lighting facility.
 35. The facilityof claim 31 wherein the lighting facility is integrated into the videodisplay facility.
 36. The facility of claim 35 wherein the integratedlighting facility is positioned to project lighting effects behind thevideo display facility.
 37. The facility of claim 36 wherein theprojected light is directed to illuminate a wall.
 38. The facility ofclaim 31 wherein the lighting facility is integrated into an audiofacility.
 39. The facility of claim 38 wherein the audio facilityincludes at least one speaker.
 40. The facility of claim 38 wherein theintegrated lighting facility is positioned to project lighting effectsbehind the audio facility.
 41. The facility of claim 40 wherein theprojected light is directed to illuminate a wall.
 42. The facility ofclaim 31 wherein the video display facility comprises a television. 43.The facility of claim 31 wherein the coordination is accomplished atleast in part through interpreting the video signal.
 44. The facility ofclaim 31 wherein the coordination is accomplished through the use of acombined signal.
 45. The facility of claim 44 wherein the combinedsignal includes lighting control information and video information. 46.An entertainment facility, comprising: a video display facility and alighting facility; and Is a coordination facility adapted toautomatically coordinate lighting effects generated by the lightingfacility with the video display facility.
 47. The facility of claim 46wherein the lighting facility is adapted to generate colors within acolor spectrum.
 48. The facility of claim 46 wherein the lightingfacility comprises an LED.
 49. The facility of claim 48 wherein thelighting facility comprises a color-controllable LED lighting facility.50. The facility of claim 46 wherein the lighting facility is integratedinto the video display facility.
 51. The facility of claim 50 whereinthe integrated lighting facility is positioned to project lightingeffects behind the video display facility.
 52. The facility of claim 51wherein the projected light is directed to illuminate a wall.
 53. Thefacility of claim 46 wherein the lighting facility is integrated into anaudio facility.
 54. The facility of claim 53 wherein the audio facilityincludes at least one speaker.
 55. The facility of claim 53 wherein theintegrated lighting facility is positioned to project lighting effectsbehind the audio facility.
 56. The facility of claim 55 wherein theprojected light is directed to illuminate a wall.
 57. The facility ofclaim 46 wherein the video display facility comprises a television. 58.The facility of claim 46 wherein the coordination is accomplished atleast in part through interpreting a video signal supplied to the videodisplay facility.
 59. The facility of claim 46 wherein the coordinationis accomplished through the use of a combined signal.
 60. The facilityof claim 59 wherein the combined signal includes lighting controlinformation and video information.